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ABSTRACT 

% The Montgomery County (Maryland) Public Schools 

(MCPS) provide English language instruction to approximately 3,800 
students with limited English proficiency. This report represents 
results of an evaluation of the English for Speakers of Other 
Languages (ESOL)/Biiingual services offered during 1981-8,2. The first 
phase focuses on the entry/exit testing process, and was designed to 
evaluate the individual instruments used for testing as well as the 
entry/exit testing process in general. The entity assessment battery 
utilized consisted of three instruments: the Language Assessment 
Scale (LAS), the Minimum English Competency Test (MEC) , and the Entry 
Teacher Evaluation interview. The exit battery consisted of a 
parallel form of the MEC, the LAS, and the Exit Teacher Evaluation. 
Major objectives of the evaluation were to determine: (1) the 
validity of the MEC as an entry/exit assessment of students* English 
proficiency; (2) whether parts of the battery are redundant; and (3) 
the reliability and validity of the overall testing process. It was 
found that: (1) test scores on the MEC and the nationally validated 
LAS are similar* suggesting that the MEC is a valid measure of 
English skills for Hispanic students in MCPS; (2) MEC scores 
successfully correlate with classroom performance; (3) the time 
required to administer the LAS is probably not justified, given its 
redundancy; and (4) regardless of their scores on the MEC, Asian 
students receive higher scores on s the Exit Teacher Evaluation and are 
often mainst reamed with less proficiency than Hispanics. These 
findings led to the formulation of several recommendations to improve 
measurement validity and reliability. (GC) 
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'EXECUTIVE SUMMARY 



' EVALUATION OF THE ES OL /BILINGUAL PROGRAM: 
VALIDATION AND RELIABILITY OF THE ASSESSMENT MEASURES 



BACKGROUND 

MCPS provides English language • instruction to students whose English 
proficiency is too limited for them^ to function effectively in regular 
classes. Approximately ;3 , 800 students most of whom ar£ foreign born, receive 
such instruction each year. "Students of limitetKtnglish language proficiency 
receive language ins true t ion 4 in English for Speakers of Other Languages (ESOL) 
and/or bilingual classes under the direction of the ESOL/Bilingual Division. 

ESOL/Bilingual services are inte'nde'd to prepare students to function with 
little or no supplemental English language instruction in regular MCPS 
classes. It is an intensive program of services that • is^ expected to be of 
limited duration for any individual student. Normally, a student with limited 
English language skills will be enrolled in ESOL/Bilingual classes upon entry 
into MCPS, receive ESOL/Bilingual services in addition to other instructional 
services until he/she reaches exit-level English language pr6ficienCy, and 
will subsequently move into a regular ' MCPS instructional program. 
ESOL/ailingual services are offered at levels ranging from beginning to 
advanced, depending on students' English language proficiency. A formal 
testing process (referred to as "entry/exit testing") has been used to 
determine students' initial placement in tt>e ESOL/Bilingual program and to 
evaluate students 1 teadiness for departure from the ESOL/Bilingual program. 

At the request of the ESOL/Bilingual Division, the Department of Educational 
Accountability (DEA) began an evaluation of ESOL/Bilingual services during the 
1981-82 school year. The first phase of this* study, focusing on ~ the 
entry/exit ^testing process, was designed to evaluate^ the individual 
instruments used for testing, as well* as the overall entry/exit testing 
procdss. This report presents'- the results of this evaluation. The second 
phase of the evaluation, an evaluation of ESOL/Bilingual service delivery and 
follow-up on program graduates, is being initiated during the 1982-83 school- 
year. , 

THE ENTRY/ EXIT TESTING PROCESS 

When students of lini'ted English proficiency enter MCPS, they are 'tested with 
a battery of instruments 'designed to measure their English language skills. 
Performance on trie assessment battery determines whether an4 at what level 
they are ■ placed in ESOL/Bilingual classes . , Students are moved from 
ESOL/Bilingual classes into a "mainstream" MCPS -instructional program when 
they attain the exit-lev-el criterion score on a similar' assessment battery. 
Whereas the timing of students' entry testing. is predetermined (upon m initial 
entry into MCPS), students may be exit tested ,at an.y point when their 
teacher(s) or parents request it to- determine their readiness for program 
exit. , { 1 



I. For Grades J-6, there are three levels of proficiency from beginning to 1 
advanced; zor Grades 7-12, 'there are five levels of prof icie*ncy\ 



THE ESOL TESTS 
*• 

Exhibit 1 describes the instruments used for entry/txit testing. The entry 
assessment battery consists of three instruments: the Language Assessment 
Scale (LAS), the Minimum English Competency Test (MEC), and the Entry Teacher 
Evaluation. * 

The LAS is a commercial test that was nationally standardized on Hispanic 
students,. It assesses receptive and expressive oral language skills and is 
individually administered. The MEC was developed by MCPS ESOL program staff 
to assess listening, speaking, reading, and writing skills. The MEC is 
generality individually adminis tered / al though parts of it may be g^roup 
administered . 

Finally, the Entry Teacher Evaluation, an interview individually administered 
by the ESOL teacher, assesses students 1 English language pronunciation, 
grammar, vocabulary, fluency, and comprehension. The Entry Teacher Evaluation 
was developed by M^CPS : ^ 

Students' .subtest scores on each instrument are weighted and combined into a 
single total entry score, which provides the basis for ' 'determining initial 
ESOL/Bilingual placement. Appendix/ A provides more detail on the test 
instruments and scoring pspcedures. m • 

The exit assessment battery also consists of three instruments: a parallel 
form of the MEC, the LAS (a parallel form does not exist), and the Exit 
Teacher Evaluation. The Exit Teacher Evaluation differs from the entry form 
in that ^it looks not only at communication skills but also places a good deal 
of emphasis on students' classroom- performance. Scores * from the separate 
instruments are' weighted to provide a single total exit score, which is then 
compared to a preestablished exit-lev^l ^cjriterion to determine students' 
readiness for departure from the ESOL/Bilingual program. The Exit Teacher 
Evaluation is usually completed by non~ES0L teachers. 

STUDY OBJECTIVES ^ 

The entry/exit testing process evaluation has three major objectives: 

o. To determine the validity of the MEC As an entry/exit assessment of 
, s-tucients 1 English language proficiency . The MEC, unlike the LAS, 
has not beer, validated on a large sample of students with limited 
» English language proficiency. 

1 To ^determine whether parts of the assessment battery are redundant . 

Elimination of redundancy would result in a more streamlined testing 
process without loss of validity* More specif ically , the question is 
whether the LAS, which is extremely time-consuming to administer, 
could be removed from tHe testing battery, 

o To determine the reliability and validity of the overall entry/exit 
testing process . Overall performance determines the placement of 
students and the services received. If the overall assessment 
battery is net valid, placement decisions are likely to be suspect 
and the resultant services received of ques t ionable appropriateness • 
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EXHIBIT 1 



KSOI. Kutry and Kxlt Test Instruments 



I nju i unit* i u 

1 atlglKlge 

Assessment 
Sea 1 4- 



Mini muni Fng I i sli 
Compel en«y 

/ 

Teacher 
Lvn I (i.i L ion 
(Kill ry) 

Teache i 
MVii 1 u;i I i on 
(Lxit) , 



Ski I Ls Assessed 



Percentage 
of Srorc »it 
Kntl y Kx it 



Mini ma 1 sound |>a i rs* 9 
I ex i cn 1 , phonemes , 
eomprebens ion f oral 
prodm i i on 

Listening, speaking 

H 

Listening, speaking, 
reading, writing 



10 



60 



J 



H) 



NA 



V routine ia I ion ^ 
gramma r , vocahu 1 a ry , 
I I ueiiey , coinprehens ion 

Kate ol learning, A 
a e ad cm i e pc r fp nuance , 
work/study habits and 
mot iv.it ion, cornel eft— ) (b) 
iiess and t ime I inevs 
ol work , e 1 ass pa r t i- 
e i pa I i on J 



Ability to eommuii 1 cat v 
with t be teacher, 
ability to eoiiuiuAni rate 
with peers, Le/ehcr's 
pred i c t i on 
s t udeii I 1 h 1 
ol sueeess 
w i t bout 1 1 
HI 1 ingtial 




NA 



(c) 



10 



30 



NA 



40 



Admin is- Grade- Form Form 
t rat ion Levels Used at Used at 
Type Assessed Entry ' Exit 



1 .rid ividua 1 



Individual 
or group 
Individual 
or group 



Individual 

(structured 

Interview) 

individual 
(teacher 
eheckl ist) 



K-6 p 
7-12' 



K-2 

3-6 
7-8 
9-12 

K-12 . 



K-12 



A, B 



(a) 



A,B 



(a) 



NA 



NA 



(a) While two forms exist, they are not parallel. Some teachers use Form A for 

Kntry and Kxit, and some use Form B. 

Q These skills are included in total score computation. 
EI^C ,,, ' UiWO «"!■*■• » U>L included In totdl score computation* * g 




SUMMARY OF FINDINGS 

The study found that the individual components of the entry/jfcxit testing 
process differed in reliability and validity and wpre to some extent 
redundant. The specific findings are discussed in the following: 

I o * Analyses comparing per f ojnnance on the MEC and LAS indicate that, in 
general, test . scores on the two. measures are similar. Sinte the LAS 
is a nationally standardized and validated measure fot Hispanic 
students, this suggests that the MEC may-be considered to be ^ valid 
measure of English language skills for the M£PS Hispanic student^. 

o Additional analyses relating MEC scores to classroom performance and 
to the performance of native Eh'glish speakers provide further 
- evidence that the MEC is a valid meafsure of English proficiency. t The 
data show that the test differentiates appropriately between students 
of different instructional levels from all language grpups, and 
measures skills possessed by the native English speaker. 

o Considerable time is devoted to the administration of the HAS, which 
must be given to students individually. However, students' scores on 
the LAS comprise only AO percent of their total scores on the 
entry/exit assessment battery* The MEC, which takes less time to 
administer than the HAS, has been found to" be a valid measure' of 
students' oral and written English* language skills and provides data 
similar to that of th/K LAS. 

o The study . found ythat regardless of performance on the MEC, Asian 
students received«?nigher scores than Hispanic 'students on the Exit 
Teacher Evaluation and because of this were sometimes exited from the 
program with l^s proficiency in the English language. This is 
primarily bemuse the score on the fl exit instrument^ which makes up 40 
percent of tfa» total exit score, reflects in large part classroom 
teachers' assessments of student performance in non-ESOL subjects. 
This finding raises questions regarding both the goals of the ESOL 
program £nd the degree to which the Exit Teacher Evaluation validly 
assesses/students 1 readiness to ^f unction in the mainstream. 

/ * ' ■' 

o The reliability cf the teaoher evaluations is at present untested. 

oweXrer, the lack of written objective criteria for their scoring 

es questions concerning trheir use. 

The reliability and validity of the overall entfry/exif process is questioned 





rem se\>pra~ perspectives. 

First, since the teacher evaluations make up 30 percent of the entry 
and + 0 percent of the exit score, problems with them strongly affect 
the validity of the overall testing process. t ~ 

Second, some teachers had a disproportionately large number.^,0f 
students with total scored of 0. This suggests that at least setae of 
the teachers are not implementing the assessment process *at entry but 
rather are simply assigning a 0 to students and placing them in the 
lowes t level of prof ic iency . 
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Third, the standard errors of the tests are large (10 to 13 points 
at entry, 6 to*7 at exit) relative to the range of scores associated 
with placement in different ins tructional proficiency levels. This 
problem is especially severe at the secondary level where a standard 
error • of this magnitude could alter a student's placement by as much 
as two levels in either direction, \ 

Fourth, for a variety of reasons, including the complexity of the 
weighting procedure, the need to combine scores from the subtests,, 
and the difficulty of scoring the LAS, computational /errors were 
found in over 30 percent of the scores. In a number at cases, these 
errors led to students being placed in an ins t/iictidtial level 
different from that actually indicated by their scores 



PROGRESS TO DATE 



/ 



As a result of informal sharing of the study resu its with ESOL program 
managers, the following changes have already^ taken place i^ the ESOL testing 
program: * \ m 



The LAS has be<en eliminated from entry and exit tesWng, resulting 
in a time savings of approximately 1,500 student hour¥\ and 1,500, 
staff hours tor the 1982-83 academic year alone. 

A testing team has been formulated by the ESOL office to tesTt all 
entering and exiting ESOL students. Prior to the 1982-83 school 
year,- this team approach was tried on a pilot basis. Results of thiV^ 
evaluation helped justify the need for the team for the overall 
testing process. The use of the tehm will eliminate much of the 
concerns associated with- inconsistent test administration and scoring 
as veil as tree core teacher and aide time for instructions 

The Teacher Evaluations now cont'ribut'e 30 percent of the total 
battery scores, and the MEC 70 percent. Thus, the total score is now 
based to a larger extent on an instrument that is valid at least for 
MCPS ZSOL population. 

Discussions have already begun among ESOL* staff at all ranks 
concerning the discrepancy in performance of Hispanic and Asian 
students cn the Exit Teacher Evaluations completed by classroom 
teachers. Plans are being madto by' both program and DEA staff to 
follow up on this findings 



RECOMMENDATIONS 

The above steps provide tangible solutions to many of the cone 



report. The following recommendations address the remaining 



n clans 
nin^c 



raised 
oncerns : 



in 



The Exit Teacher Evaluation .process needs further examination. As 
it stands, 30 percent of the student's exit score is based on 

directly related 
ssue of whether 

should override ^a student's opportunity to be mainstreamed from a 
language pro gran must be addressed. And, if it is decided that this 



considerations not 
The phi losophical 



o English language proficiency, 
classroom performance can or 
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performance is an important criterion for leaving "the ESOL program, 
the services provided' to students need to be examined to assess their 
adequacy in this area. 

A controlled reliability analysis is suggested for what remains of 
the entry and exit batteries* ^Included in this analysis would be an 
examination of the teacher evaluation instruments in light of their 
objectivity and appropriateness Tor the purposes for which they are 
used , a complete item analysis of the MEC with a view towards 
possible elimination of some items, and a Rasch calibration of items 
across grade and" language proficiency levels. 

Every effort should be made to further reduce the -measurement error 
associated with the entry and exit batteries. .As a short-term 
solution, reduction of the number of ESOL working- fevels In Grades 
7-12 from five levels to three ,' thereby widening the range of scores 
in earth level, would tfffset the problem, of unreliability o"f placement 
of students in th^ levels. . Furthermore, since staff report that 
combination classes are now frequently used because there are not 
enough students at a single level to constitute' "an entire class, this 
reduction in thp number of levels would serve a practical purpose as* 
Well. 
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MCPS provides English language instruction to st.udents .whose English 
proficiency is ' -too limited for them to function effectively jftx regular 
classes. Approximately 3,800 students, n}ost of' whom are foreign born, receive 
such> instruction each year. Students' of limited- English language proficiency 
^zg&eive language instruct ion in English for Speakers of Other Languages (ESOL) 
> and/or bilingual classes under the direction of the ESOL/Bilingual Division. 

ESOL/Bilingual services 'are intended to prepare students to function with 
little or no supplemental English language instruction in regular MCPS 
clashes. It is an intensive program'of services that is expected to be. of 
limited duration for any individual student. Normally, a student with limited 
English-language Skills will be enrolled in E^OL /Bilingual classes upon, entry 
into MCPS , receive ESOL/Bilingual services in addition to v other instructional 
services until* he/she reaches exit-level English language proficiency, and 
will ; subsequently move into a ^regular MCPS. instructional program. 
ES0L/3ilingual services are offered, at levels ranging from beginning to 
advanced, depending "on , .students 1 English language proficiency. Vformal 
testing process (referred to as "entry/exit testing") has been used to 
determine students' initial placement in the SSOL /Bilingual program and to 
evaluate students 1 .readiness for departure frcfrt the ESOL/Bilingual program. 

\ ... . '' ■ 

At the request of the ESOL/Bilingual Division, the Department of Educational 
Accountability. (DEAK began^ an evaluation of ESOL/Bilingual services during the 
1981-82 school year. # The' first phase of this study, focusing on the 
encry/exit testing process -was designed to evaluate the individual 
instruments used for* testing,- as well as the* overall entry/exit testing 
process.. This report presents the results of this evaluation. The second 
phase of the evaluation, an evaluation of ESOL/Bilingual service delivery and 
follow-up on program graduates, is being initiated during the 1982-33 school 
year. 

THE ENTRY /EXIT TESTING PROCESS ' 

When students of limited English proficiency enter MCPS, they are tested with 
a battery of instruments designed to measure their English language^ skills. 
Performance on the assessment battery determines .whether- and at what level 
they are placed, in ESOL/Bilingual ^classes. Students are moved from 
ESOL/Bilingual classes into a "mainstream' 1 MCPS instructional program when 
they^ attain the exit-level criterion score on a similar assessment battery. 
Whereas the timing of students' entry testing is predetermined (upon initial 
entry into' MCPS), students may be exit tested at any point when their 
teache.r(s) or parents request it to determine their readiness for program 
exit. < 

* 

* » 

1. For Grades 1-6, there are three levels of proficiency from beginning to 
advanced; for Grades 7-12, there are five levels of proficiency. 
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THE ESOL TESTS , - 

Exhibit 1. describes % the 'instruments used for entry/exit testing. The entry 
assessment ^battery consists of 1 three instruments: the Language Assessment 
Scale (LAS), the Minimum English Competency Test (MEC) , and. the Entry Teacher 
Evaluation. , " - " 

.The LAS is a commercial test that was nationally standardized on Hispanic 
students. It assesses receptive and expressive oral language skills 4 and is 
individually administered. The MEC was developed by MCPS ESOL program staff 
to assess listening, speaking, reading, and writing slcills. The MEC is 
generally individually administered , although parts of it may be group 
administered. 

Finally, the Entry Teacher Evaluation, an interview individually administered 
by the ESOL teacher, assesses students 1 English language pronunciation, 
grammar, vocabulary, fluency, and comprehension. The Entry Teacher Evaluation 
was -developed by MCPS. 

T r < ' - 

Students'/ subtest scores on each instrument are weighted and combined into a 

single total entry score, which provides the basis for determining initial 

ESOL/Bilingual placement. Appendix A provides more detail on the test 

instruments and scoring procedures. 

1 

The exit assessment battery also consists of three instruments: a parallel 
form of the MEC, the LAS (a * parallel form does not exist), find the Exit 
Teacher Evaluation. The Exit Teacher Evaluation differs from the entry form 
in that it looks not only, at communication skills but also places a good deal t 
of emphasis on students ' classroom performance. Scores from the separate 
instruments are weighted to provide a single total exit' score, which is then 
compared to a* preestablished exit-level criterion to determine students' 
readiness for departure from the ESOL/Btlingual program. The Exit Teacher 
Evaluation is usually completed by non-ESOL teachers. , 

STVLY OBJECTIVES 

The entry/exit testing process evaluation has three major objectives: 

o To determine the validity off the MEC as an entry/exit assessment of 

students' English language proficiency . The MEC, unlike the LAS, 
has, not beer, validated on "a'large sample of students with limited 
English language prof ic iency < 

o To* determine whether parts of the assessment battery are redundant . 
Elimination of redundancy would result in a more streamlined testing 
process without loss or validity. More specifically, the question is 
whether the LAS, which is 'extremely time-consuming to administer, 
•could be removed from the testing battery. 

o • To determine the reliability and validity of the overall efttry/exit 
testing process . Overall performance determined the. placement of 
students and the services' received. If the overall assessment 
battery is not valid, placement decisions are. likely to be suspect 
and the resultant* services received of questionable appropriateness. • 
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KSOL Kntry and Exit Test Instruments 



1 ns 1 1 (iiiu ii t 

La nyn. i ge 
Assessment 
Scale 



Ski! 1 s Assessed 



I'ei cent age 
o,f Scoi e at 
Kntry Exit 



Adminis- 
C ration 



Grade 
Levels 
Assessed 



Form 
Used at 
Kntry 



Minimal sound pairs. 
Icx ica I , phonemes, 
comp r ehous ion , ora 1 
|»f oduct ion 



10 



Minimum Kuglisji Listening, speaking ^ 
Cuiiipc t enc v 

Test Listening, speaking, 

reading, writing 



60 



Tear he i 
Kva i ii. 1 1 i on 
(Kul i y) 

Teaelu- 1 

Kya I uat 1 on 
' (Kxit) 



J 

l A i onunc ial Ion, * 
gramma r , vocahn I ary^ 
I luency, comprehension 



Kale ol learning, m *\ NA 
academic performance , 
woik/smdy habits and 
motivation, complete- > (b) 

ness and t imeliness 

if 

t>i woik, Mass parti- 
c i p. it Ion ' • . 

' \ 

Ability i.u communicate 1 
wllli the teacher, 
ability to communicate 
wi tli peers, teacher's 
prod let Ion of t he 
student 1 m I ikel ihood 
ot hi ice ess in school 
without further KSOL/ 
Bilingual, services 



NA 



(e) 



10 individual - 



50 individual 
or group 
Individual 
or group 



NA ; Individual 
^structured 
interview) 

AO Individual 
(teacher 
checklist) 



K-6 
7-12 



K-2 

'3-6 
7-8 
9*12 

K-12 



K-12 



(A) 



NA 



(a) While two forms exist, they are not parallel. Some teachers use Form A for 

Kni i y .aid Kxit, and some uoe Form b. 
J ^O" iese skills are Included in total score computation. 
ERJC*'" 0 skills are not included in total, score computation. 



• m Form 
Used at 
Exit 



(a) 
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o Third, the standard errors of the tests are large (10 to 13 points 
at entry, 6 to 7 at exit) relative to the range of scdres associated 
with placement in different instructional proficiency levels. This 
problem is especially severe at the secondary level where a standard 
* error of this magnitude could alter a student's placement' by as much 
as two levels in either, direction. 

o Fourth, for a variety of reasons, including the complexity of the 
weighting procedure, the need to combine scores from the subtests, 
and the difficulty of scoring the LAS, computational ferrors were 
found in over 30 percent of the scores. In a number of cases, these 
errors led to students being placed in an instructional fc level 
different from that actually indicated by their score. ^. 



PROGRESS TO DATE 

As a result of informal sharing of the study results with ESOL program 
managers, the following changes have already taken place in the ESOL testing 
program: 

o The LAS has been eliminated from entry and exit testing, resulting 
' - in a time savings of approximately 1,500 student hours *• and 1,500 

staff hours for the 1982-83 academic year alone. 

o A testing team has been formulated by the ESOL office to test all 
entering and exiting'ESOL students. Prior to the 1982-83 school 
year, this team approach was 'tried on a pilot basis. Results of this 
evaluation helped justify the need for- the team for the overall 
testing process. The use # of the team will eliminate much of the 
concerns associated with inconsistent test administration and scoring 
as well as free more teacher and aide time for instruction. 

o The Teacher Evaluations now contribute 30 percent of the total 
battery scores, and the MEC 70 percen-t. Thus, the total score is now w 
based to a larger exten,t#Qfi an .instrument that is valid at least for 
MCPS ESOL population. 

o Discussions have already begun among ESOL staff at all 'ranks 
concerning the discrepancy in performance of Hispanic and Asian 
* students on the Exit Teacher Evaluations, completed by classroom 
teachers. Plans are being made by both program and DEA staff to 
follow up on this finding. 
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RECOMMENDATIONS 



The above steps provide tangible, solutions to' many of the concerns raised in 
this report. The following recommendations address the remaining concerns: 

o b The Exit Teacher Evaluation* process needs further examination. As 
it stands , 30 percent of the student's exit score is based on 
considerations not directly related to English language proficiency. 
The philosophical issue of whether classroom performance can or 
should override a student's opportunity to be mainstreamed from a 
^language program must be addressed. And, if it is decided that this 
performance is an important criterion for leaving the ESOL program, 
the services provided to students need to be examined to assess their 
adequacy in this area. 

o A controlled reliability analysis is suggested for what remains of 
the entry and exit batteries. Included in this analysis would be an 
examination of the teacher evaluation instruments in light of their 
objectivity affd appropriateness for the purposes for Which they are 
used, a complete item analysis of the MEC with* a view towards, 
"possible elimination of some items, and a* Rasch calibration of items 
across grade and language proficiency levels, 

o Every effort should be made to further reduce the measurement error 
associated with the entry and exit batteries. As a short-term 
solution, reduction of the number, of ESOL working levels in Grades 
7- 12, from five levels to three, thereby widening the range of scores 
in each level, would offset the problem of unreliability of placement 
of students in the/ levels. Furthermore* since «taff report that 
combination classes are now frequently used because there are not 
enough students at a single level to constitute an entire class f this 
reduction in the nuiroer of levels would serve a ftoactipal* purpose as 
well. I . 

The remainder of this document \>re*ent9 the detailed f indings. The general 
raauics are contained in the bbtfy of the text; the more technical information 
supporting these findings is presented in the footnotes and appendices. 



DETAILED FINDINGS 

THE STVDY SAMPLE 

2 

Dunns the 1930-31 academic year, 3,746 students were enrolled in the ESOL 
progr^r. for part or ail of the year. Normally, students are entry tested only 
cnce to determine thefr initial placement in the ESOL program and are exit 
tesced only when they are considered as candidates tor mains tr earning in a 



2. Thi3 figure includes all students who participated in E^)L classes" at any 
time during 1930-31. The maximum number enrolled at any p#"int in time was 

N 3,20O. 
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regular MCPS instructional* program. To provide the necessary data for this 
study, however, all students who were enrolled in the &SOL program in the fall 
of 1980 (2,593 students) were given the entry test battery. Similarly, all 
students who were enrolled in the ESOL program in the spring of 1981 (2,507 
Students) were given* the exit test battery regardless of their perceived 
readiness for departure from ESOL. For purposes' of this study, .students who 
had incomplete, zero, or otherwise erroneous test batteries were eliminated 
from the sample. This left a total of 2;376 entry tests and 2,406 exit 
tests, representing 63 percent and 64 percent, respectively, of the population 
of students who were enrolled in the ESOL program for at least part of the 
1980-81 academic year. Unless otherwise notfcd, the results reported in the 
subsequent sections are based on this sample of students. 

It should be stressed that this sample is more or less'" representative of 
students enrolled in ESOL during 1980-81, but is not representative of the 
smaller number of students who either entered the ESOL program for the first 
time in the fall of 1980 or exited from the £S0L program in the spring of 
1981. In fact, many students who are included in] j this sample entered ESOL 
prior to 1980-81,, thus inflating the entry test score averages, over what would 
be expected from true, program entrants. Similarly, many students we>re exited 
from ESOL prior to the end of the 1980-81 school' year and were not part of the 
sample included in the exit testing. Therefore, exit test score averages 
obtained from this sample would be lower than those expected from bonafide 
candidates for exit. 

To facilitate comparisons among students', average scores on the separate 
tests, students' scores for some analyses were transformed into standardized 
scores. For each test at each level (Grades K-2, 3-6, 7-8, 9-12)/ students' 
original scores were rescaled into scores with a mean of zero and a standard 
deviation of .one (known a& 2 scores) . Such a transformation yields scores 
with the same cietric that can be straightforwardly compared without a loss of 
original precision. 



3. A student can receive a- total entry or exit score of zero only if he/she 
receives, zero- scores on each of the separate tests. In theory, this should 
occur only for students with no English language skills. 'However, 187 (8 
percent) of the students who were entry tested had total scores of zero. It 
is suspected chat at least some or these 2ero scores were erroneous for two 
reasons. First, not all of the students with zero scores were in the 
beginning ESOL instructional levels at the tine of testing. Second, some 
ceacher3 had a disproportionately large number of students with zero scores, 
suggesting that at least some of the students were assigned across-the-boarrd 
zeros rather than being entry tested. Since it was impossible to separate the 
erroneous total zeros from the valid ones, all students with zero total entry 
scores were eliminated from the analyses. 

<*. Standardized 3cores are computed by subtracting the group mean front each 
student's rav score and dividing that number by the group standard deviation* 
In this metric, for example, a score of 1.0 i3 equivalent to one standard 
deviation above the me$n> and a score of -1.5 is equivalent to one and 
£ae~haif standard deviations below the mean. 
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VALIDITY OF THE MEASURES 



Three kinds of validity were examined in this stu£y: content validity, 
concurrent validity, and construct validity. Procedures for assessing these 
.are described briefly in the following: 

The analysis of content validity examined the English ianguage 
skills and competencies associated with high performance on , the 
test(s). 

Goncurrent validity was established by comparing students' 
performance on the instrument(s) to a predetermined standard (e.g., 
performance on -an alternate instrument whose reliability and validity 
are established) . 

A comparison of students' performance on the testes) to the way 
things are expected to operate in the- "real world" 4 was used to 
establish construct validity . Students' test scores, for' example, 
should improve as they progress through the levels of ESOL/Bilingual 
instruction. * 

» * + 

THE MEC . * * • 

The validity of the MEC was a major consideration sirfl;* it is a MCPS-developed 
instrument that had not been subjected to the extensive reliability, validity, 
and norming analyses that are common for commercially developed instruments 
prior to their use. The LAS, *it will be recalled, is an ins trumtnt/whose 
reliability and validity as ' a measure of English language skills was 
established on a national sample of , Hispanic students. The first step in 
establishing the validity of the MEC, then, was to compare students' MEC and 
LAS* scores. The study found that results obtained from MEC and LAS testing 
were very similar for most groups of students ,^ which established the 
concurrent validity ©f the MEC for Hispanic students. 

The construct validity of the MEC was analyzed by charting students' 
performance oa the MEC as they progressed through the levels of ESOL/Bilingual 
services. Students'' placement in levels of English language instruction 
(ranging from beginning to advanced) ia supposed to be determined by their 



5. Correlations between the total LAS and total MEC scores ranged betveen, .74 
and . 7? for entry testing results and between .56 and .77 for exit tasting 
results. Correlations between the LAS Total scores and MEC Listening/Speaking 
scores were of a comparable magnitude, while correlations between LAS Total 
scores and the MEC Reading or Writing scores were lower. This pattern' is 
expected since :he LAS only measures oral English skills. 

In addition, when the mean LAS and MEC scores at entry and exit were graphed 
by 1S0L instructional levels, the curves wete very similar for both tests, 
indicating equivalence of the measures for the groups tested. The correlation 
arc 4 graphical findings both support cha concurrent validity of the M.EC (sec 
riguras 3-L through 3-4 of Appendix 3). 
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fnglish language proficiency. The study found that students ^ assignments to 
levels of English language instruction were associated with their MEC scores; 
that is, the average MEC scores for students within ESOL/Bilingual lev 
increased for each level from beginning to advanced. In additionT 

students' MEC scores increased significantly over , a year's time in thi 
ESOL/Bilingual program, ' Both sets of analyses uphold , the construct 
validity of the MEC. 

The primary emphasis of ESOL/Bilingual instruction varies somewhat by grade 
level's and by levels of English language profipiency. Instruction for 
students in the primary grades and those, at beginning ESOL/Bilingual levels 
emphasizes listening anti speaking skills, whereas instruction- for students who 
are more advanced in their English language skills and/or at higher grade 
levels emphasizes reading and writing skills. Scores obtained on Che MEC 
confirmed these expected difference's; That is,, in the elementary grades It 
was found that there were significant diff eren ces be.tw.eert beginning' and 
intermediate s tudents^Ln_£hs-4^ speaking s e1:t I5f^ 

exited 'students differed significantly in reading and writing. 'By Graded ",' 
7-12, however, there was a shift with significant differences observed" between 
beginning and -intermediate students' reading and writing skills. No 
differences in skill level were found between advanced and exited students iri- 
Crades 7-12 (see Figures; B-5 and B-6) . These differential patterns of 
increased, performance correspond co* differential emphases in ESOL/Bilingual 
instruction, which provides further evidence for the content and construct 
validity of the MEC,. 



Finally, a sample 
MEC in the spring o; 



of 19^ native English-speaking students were tested on the 
1982, Almost all of the students (97 percent) s 



6. The one exception to this pattern is that students at Level 6 had a' lower 
mean MEC^ Reading score at entry than students at Level 5 in Grades 9-12. For 
more detail, again refer to Figures B-l through B-4. 

7. This finding is based on an analysis of differences in entry and exit test 
scores from the fall of L 980 and the spring of 1981 testing for the 38 percent 
of the T»ain sample who were enrolled in the ESOL program at both points in 
time,, eliminating students vho had zero total scores at either testing. The 
gains over the course of the 1980-81 year were quite large, and statistically 
significant, for all of the MEC scores as well as the LAS and total scores, 
across all grade levels. See Table 3-3 for details. 

8. these results were obcained from a Srtmple of 92 students enrolled in ESOL 
in the spring of 1982 or who had recently been exited from ESOL classes. The 
students were selected at random to represent tlffe various levels "of ESOL 
Languaga competency and to represent a variety of grades. The MEC Form A was 
administered to beginning and intermediate! students", vftile Forii B was given to 
advanced and,e::iced students. Results wer£ not" compared across forms. 

9. These students were drawn from sevap schools representing varyjji£-l*v*ts- 



of average academic performance an 4 d were enrolled in Grades I through 8 and 
II. They were tested with Form A of tlte MEC. These students represent the 
standard of English language competency ag4inst which* the performance of ESOL 
students is compared, and therefore t^ey should attain 3cores which would 
indicate r.o need of ESOL services. & 
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received total scores at or beyond tl^yexit-le^el^ri terion score (see Tatjle 
B-4) • Since attainment # of thev exit-^level criterion indicates a sufficiently 
advanced level of English language competency for performance in. the 
mainstream, this finding Jends *' support " to the' concurrent and construct 
validity of the MEC. ■ ^ * 

For all of the above" reasons, the MEC may be considered to be a valid measure 



o& English language skills for the MCPS ESOL/Bilingual population. 



THE LAS • . • - * ' . - 

The\LAS has been " validate^ ancL wormed on a national sample of Hispanic 
studenH ^, b ut its Y^i^ity as a measure of English language proficiency for 
speakers^ "WS^^he r native languages has not been demonstrated. It is a test of 
^receptive /and expressive oral language skills, and, .thus prdVides.no measure of 
students' reading and -writing skills, which are equally \ important /for 
effective functioning in regular classes. The emphasis on oral language >4lso 
may bias the test for certain language groups. It is alleged, for example* 
-that some of the items requiring discrimination in pronunciation are biased 
against Asian students, whose native languages are based integrally on tonal 
inflections. For these reasons, the content and cbnstruct validity of the LAS 
as a test of general English language proficiency for. the MCPS ESOL/Bilingual 
population is in ques'tion. - 

The computation of total LAS scores is a comply process, involving conversion 
of raw scores to table values in three decimal places ancl hand computation 0f 
a total score in three decimal places. Many computational errors are made by 
ESOL teachers and aides, resulting in scores which exceed the accepted score 
range, /In short, although correctly computed LAS total scores may be 
reliable, the considerable amount of error that is introduced in practice in 
the scoring process results in unreliable measures. (See Appendix A for 
details.) 

Finally, it has already been shown that individual students' scores, on the MEC 
and the LAS are very similar (the correlations between LAS and MEC scores are 
fairly high, ranging between .74 and .77 at entry and between -.56 and .77 for 
exit; again see Tabies 3-1 and 3-2)'. A comparison of average MEC and LAS 
scores,, by grades and ESOt levels provides further evidence for the high degree 
of overlap between the MEC and the LAS (see Table B-5) . Across all grades and 
levels of ESOL instruction, students' mean LAS and MEC scores were very 
similar. Ic is apparent that the LAS provides very little information over 
and above the results' based on MEC testing. 



ERLC 



While the LAS has been nationally validated and standardized on Hispanic 
students, its use in across-the-board assessment Of all 'ESOL students is 
questionable. Additionally, while considerable time is devoted to the 
individual administration of the LAS, students' scores on the 'LAS comprise 
only 10 percent of their total sqores on the. entry/exit assessment battery. 
Moreover, the MEC has been found to be a valid measure of students' oral and 
written English language skills, and it assesses a broader demain^of 3kill 



area than does the LAS. Eor these reasons, the LAS is superfluous in \the ESOL 
entry/exit testing process . 
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THE TEACHER EVALUATIONS 




to _£he— 
ucion to 



acherr^Evfflu 



aluatians make Che largesc 

student's total encry or exi/t Cest score. (In 1980-81, . th$ 
and che Teacher Evaluation made „a 60 percent and 30 percent contribution^ 
respectively, to' students f total entty scores, and a 50 percent and 40 percent 
contribution, respectively, to students' /total exit scores.) The . Entry 
Teacher Evaluation fcsM>ased on teacher ratingjs of student performance on a 
structured interview in five separate area^. Tha Exit Teacher Evaluation, oi\ 
the other hand, is a scale af ratings on, aspects of students' classroom 
performance "such as- completeness and quality of work, study skills, and work 
haMts. : Exit teacher ratings are averaged] across all >; teachers (ESOL and 
non-ESOL) by student, whereas enfry ratings *re made only by the ESOL teacher.. 

Analyses "N^how that' the^ performance ratings which comprise a major* portion # of 
the Exit Tocher Evaluation may be x affecting 1 the scores of some groups of 
students mo^e than others.. As a group, Hispanic students, for example, 
generally ^obt 4 a^n performance ratings from th^lir teachers that are lower than 
thei* MEC secures, while Asian students dend to receive performed!* ratings 
that are higher chan their MEC scores ff&e Figures 3-7 through B-10) . At 
a result, Hispanic students tend ' to be retained In ESOL/Hilingual classas 
longer than they wduld if their exit .scores I were determined solely on tha 
basis of their MQC and LAS performance. Asian students, oh the other hand, 
tend to leave ESOL c\agses -with lower English I language skills because teachers 
perceive their class\oom performance in a morte positive light.* Whether or not 
these differences are reflected in performance! once the student is returned to 
the mainstream settina is not known at this pbint. It is- clear, however, that 
combining the language J^nd performance scores into one score at exit obscures 

and may be leading to ^in&ppropripte decisions 



these profile differences 
regarding whether or not thti student Is ready to 

classroom, , \ 



function in the mainstream 



These findings call! to the fore some fundamental issued regarding the teacher 
evaluations r especially tha format used for exit! testing. Specifically, # the 
inclusion of classroom performance* items on' the e3it evaluation raise* 
critical questions concerning the goals of the S^OL/ Bilingual program. Is the 
prbgram intended to affect student- dfevelopmenit and classroom* performance as 
veil as language proficiency? And, if it is, 1 are th« services that are 
prqvided appropriate for addressing such performance gaalp' ) 

1 % 

Finally, it must be mentioned that it is not jinovn hew reliable the ratings! 

arel 



either at entry or exit, as no objective beh^viora] 



10. 



criteria are provided 
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More specifically, average standardised scores on the MEC total from exit 
and the Exit Teacher Evaluation were calculated £,or Asian and Hispanic 
by grade levels and levels of ESOL instruction/ With the exception 
6 students in Grades 7*3 and 9*12, Asian students' mean MEC totals 
;s lower then their mean Exit Teacher Er/aluation totals. For 
students, there were no major differences * between mean MEC and 
valuation tqtals at the lower E50L levels, but at. the mora advanced 
els across grades, students' rsean Exit Teacher Evaluation tocaia were 
lover than their mean MEC totals.*-* Thest Results are shown in 



•7 through 3-10. 



26 



for the assignment of ratings. Thus., ' it is not known whether ratings produced 
by different teachers wotxld be equivalent as regards, either language- 
proficiency or behavior. ' 

♦ ■ a ' • ' . • •/• •• •. 

I 

In. summary; the Entry Teacher Evaluation probably .introduces a source of 

unreliability iQito.,tfae entry^gifrsting process, and the Exit Teacher ^Evaluation 

reduces the' validity of. the exit assessment procedure as a measure of English 
language' prof iciency; \ * , 7 

• " . - ■ - . " ■' ■ ■' ..4 

VALIDITY OF THE OVERALL E^TRY/EXIT TESTING PROCESS * 

The study also examined the validity of the total scores on the entr^r and exit- 
test batteries, - particularly in terms of their usefulness for purposes of 
assigning students to' levels of ' ESOL instruction and determining their 
readiness/for termination of ESOL services. * 

The problems discussed above; especially « those concerning the teache* : 
evaluations, clearly affect the usefulness of the overall entry/exit testing 
process** • In addition, however, 0 there are three 'other sources of weakness that- 
challenge the reliability and validity of the overall . procedure . . 

First, one problem with the reliability of the entry testing process is that 
relatively large proportion of the total entry, test, scores were zero (8 
percent). It is possible for students with absolutely no English language 
skills to obtain; total entry scores* of zero; yet this figure seems excessively 
large. An analysis of entry- test scores assigned by individual teachers 
showed that some teachers had a disproportionately large number of students 
with 'total scores of zero, suggesting that 'at least some of the zero entry 
scores, may be attributable to a practice of nontesting at»entry on the part of 
spme teachers. .This practice, if it occurs, reduces the reliability "of the 
entry testing process. 

' % \ ; • ■ t ' , ■ , ' . . \ < •• \ 

Second, there is also reason to believe that the measurement .- errors ~of both 
entry and^ exit total batteries -are large.' The standard deviations of 'the 
total scores are large (see Table B-6) . If we assume that v the .total score 
reliability is in the s neighborhood of ;8 (a fairly high reliability 
coefficient' for tests of bheSe types) then the standard/errors of measurement 
* range from 10 to 13. points for the entry total scores arid from 6 to 7 points 
for, the exit total scores • (see Table B^6) . These numbers are large relative 
to the range of scores associated with placement in specific ESOL 
instructional 'levels (see Table B-7) . . % - 

£ standard error*e^f 10 means that, upon retesting, approximately one-third of 
the students would be expected to obtain scores more tljan 10 points higher or 
lower than their original scores purely by* dhance. /For secondary (school j 
students, where a student could be assigned to any 'one of five levels, score 
variations of this magnitude could altfer a . student r s entry £S0L placement by 
as xuch as two levels-in either direction. This suggests that the total test 
score discriminates poorly for purposes of establishing placement level^ that 
differ in terms of English language proficiency f-Qr. students in Grades 7«-i2. 

For all o& the abpve reasons, a student yho is reassessed on the test battery- 
would be almost as likely to be placed in a different ESOL/Bilingual placement 
level as the same .one due to sources of, unreliability .and invalidity in the- . 
total tasting, process rattier ..than -to students' English language competency. . 



Third, for a variety of reasons, including the complexity of the weighting, 
procedure the need to combine scores front; the subtests, and the difficulty of 
scoring the LAS, computational errors were found in over 30 percent d>f the 
scores. In a number of (cases, these, errors led to students being placed in an 
instructional level different from that: actually indicated by their score'. 

»Thus , the overall entry/exit assessment procedure has some critical weaknesses 
aftd needs improvement before it can be considered a reliable and valid tool 
for either assigning students to ESOL/Bilingual instructional levels or 
determining whether a. student may return to the mainstream. 

' * ACCOMPLISHMENTS TO DATE 

- -;T 

Based upon the informal sharing o£ preliminary evaluation findings with the 
ESOL/Bilingual program managers, 'several substantial changes which will 
greatly enhance the ESOL/BilinguaL testing process have sfiraady.been made. To 
reiterate: 

o The^LAS has be ( en eliminated from entry and exit testing, resulting 
• ? in* a time savings of approximately 1,500 student hours and 1,500 
starff hours for the 1982-83 acad.emic year, alone. 

o A testing team has been formulated by .the ESOL, office.. This ' team 
will conduct all ESOL testing for the 1982-83 year. The (Rse of 
the team will eliminate much of the concerns associated,, with' 
inconsistent test administration and scoring as well as free more 
teacher and aide time for instruction, 

o The Teacher Evaluations now contribute* 30 percent of thg total 
battery \ scores, and the MEC 70 .percent. Thus, the <total score is now 
based to a larger extent on an instrument that is valid at least for 
the MCPS ESOL population. * 

' ' * -4 

• - ! 4 

o Discussions: have already begun among ESOL staff • at all ranks. 
'•}■■ \ concerning the discrepancy in p^e-rf ortnance assigned ta , Hispanic and 

. Asian students on the Exit Teacher Evaluation by classroom teachers. 
Plans are being, made by both program and DEA staff to follow up on 
this finding. * 

DEA, will continue to /work closely with the ESOL/Billngual' Division in the 
conduce c*f their program evaluation/ Additional Activities, * which are planned 
to start in the. 1982-83 school year, include a follow-up evaluation .-oft former 
ESOL students who were mains t$eai!ied into regular MCPS glasses-. It is 
suggested that future activities also include the recommendations contained in* 
this document for further study of the test batteries. 



It. This team has been operating cn a . pilot basis for approximately two 
years. Findings included in this report helped substantiate the need far this 
team in ail ESCL' ancry and exit testing. ~ • 
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THE ESOL TESTING PROCESS 



THE ESOL TESTS ' 

Four'dif ferent instruments are used in the ESOL Entry and Exit Teacher 
assessment process: the Language Assessment Scale (LAS), published and 
standardized by Linguametrics Group; the Minimum English Competency Test (MEC) 
* developed in MCPS; a teacher evaluation of student language- prof iciency 
(structured interview) utilized at entry; and a teacher checklist of student, 
performance utilized at exit. Both teacher instruments were developed in 
MCPS\ Exhibit A-l illustrates these tests, the skills they assess, and the 
levels and forms available. 

tt * 
.. * . i 

Examination of Exhibit A-l indicates that only one of the four measures, the 
MEC, may be administered in a group setting,' although it is usually 
individually administered. The MEC is primarily multiple' choice. The^IsAS 
must be individually administered since it assesses solely receptive and" 
productive language. It is time consuming to administer and for this reason 
is not favored by many teachers. * 

The teacher evaluation used at entry is a structured interview and is 
individually administered. . The results are somewhat subjective since the 
teacher must rate the student's performance in five aspects of English usage 
on a six-point rating scale from beginning language skills to native speaker. 

The teacher evaluation used at exit testing is a checklist of performance 
completed by the student's 'ESOL and non-ESOL teachers. (The three instruments 
illustrated above are administered or completed solely by ESOL teachers or 
aides.). At exit', all of the student ' s teachers rate him/her on eight .aspects 
of behavior, of which only the first five are included in the student's exit 
score (see Exhibit A-l). Ratings assigned by all the teachers are averaged 
into one score ^ per behavior area, thus producing scores that are less 
dependent on Che Subjective evaluation of a particular teacher than is' the 
case at entry testing/ 




EXHIBIT A-l 



ESOL Entry and Exit Test- Instruments 



1 1 nst ruiiiLMitj 
♦ 

Assi'iismeut 
Sea I u 



Ski! Is Assessed 



^Adminis- Grade 
tration Levels 
Type Assessed 



'Minimal sound pairs," 
lenicaJ , phonemes % 
comprehension, oral 
production 



Minimum Kngl fsh Listening, speaking 
Competency 

Tent t Listening, speaking* 

reading, writing 



Teacher - * Pronunciation, 

[.valuation , grammar, vocabulary, 
(ten try) fluency, comprehension 

TtMchei Rate of learning, , 

Iwa I nation academic performance, 

(Exit) work/study habits and 

motivation, complete- 
ness and timeliness 
of work, class parti- 
' cipation J 



' (b) 



Abilixy to communicated 
with the teacher, 
ability to communicate 
w^th peers, teacher's 
prediction of the 
student's likelihood 
of success in school 
without further ESOL/ 
Bilingual services J 



> (c) 



Form Form • 
Used at Used at 
Entry Kyit 



Individual 



Individual 
or group 
Individual 
or group 



Individual 

(structured 

interview) 

a 

u 

Individual 

(teacher 

checklist) 



K-6 
7-12 



K-2 

3-6 
7-8 
9-12 

K-12 



K-12 
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A,B (a) A,B (a) . 



NA 



(a) While two- forms exist, the.y are not parallel. 

O „^utry and exit, and some use Form B. 

:KJC [ bese skills are included in total score computation. 

^Wf^Jhese skills are not included in total score computation* 



Some teachers use Form A for 

best csf i mum 
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THE SCORING PROCESS „ 

Each of the four instruments has its own scoring algorithm. Exhibit A-2 
illustrates the processes by which subscale scores are obtained, and 
Exhibit A-3 shows final computations for the LAS, MEC, Entry/Extt Teacher 
Evaluations, and total score. 

It may be noted that the LAS, which initially "princes a maximum totaj score 
of 101 at Grades 'k-6 and 95 at Grades 7-12,- contributes on^y 10 percent of the 
total score upon which ESOL program placement is based. The MEC, which 
produces a maximum total spore of 40 at Grades K-2, 100 at Grades 3-6, 130 or 
180 at Grades 7-8 , and ^80 at Grades 9-12, contributes ,60 percent of the 
total placement scores Finally, the teacher evaluations,- which produce 
maximum total scores of 30 and 132 at entry and ex^t , respectively , contribute 
30 percent of their respective total scores. Thus, it- may be concluded 
that a students placement in the ESOL continuum of services' is largely 
influenced by performance on the MEC and by Entry/Exit Teacher* Evaluation. 

PROGRAM PLACEMENT 

* . 

Program placement is determined primarily. by the student's total score on the 
ESOL entry or exit test battery. However, the ESOL teacher is '* empowered to 
adjust the student's placement level up or down either if it is believed that 
the test results are inaccurate or if otKer factors, such as student behavior 
or, motivation, are considerations. Exhibit A-4 displays the placement levels 
and score ranges utilized in the provision of Ej!0L/ Bilingual services in 
MCPS. Students in Crades K-2 have two working levels in ESOL into which they 
may be placed, students in Grades 3-6 have three working levels, and students 
in Grades 7-12 have five levels. 



lN.See the Column labeled ,f Raw Score Maximum" in j&xtfibit A-2. 

.... > / 

2. L30 at entry, 180 ac exit. ^ ^ ^ 

, * , rf ■ 

3. In 1980-81, the MEC contributed 50 percent of 'the to-Tal-exit score. This 
was changed to 60 percent in 1981-82. t 

4. The Exit teacher evaluation contributed 40 percent of the total exit score 
in 1930-81. This was changed to" 30 percent in 1981-82. 
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EXHIBIT A-2 
Scoring of £SOL £ncry and Exic* Inacrueancs 



Inscruaanc trades 



Subcase 



Raw 
Scora 

Maxlaua 



Convsrsion 
Algorithm 



Convarcad 

Scora 
Maxlaua 



~anguag* 
Asaassasnc 
Scaia 



Ur.guaga 
Assaajran: 

5*:aia 



Ur.irua 
Zr.glisn 

Tasc 



Mlmeal sound pairs* 30 

Laxical 20 

Phonsaas 36 
Sancanca 

2oaprah*naion jO 

Oral production * 5 



Mir.iaai sour.d palra Zu 

laxical 20 

Phonaaaa 36 

Casprafanaion 10 
)ral production 



Liscanlng 

Llscaning/spaaking 
Spsaking 



13 



3-o 



Lis caning > \S 

'-Ucanir.g; asaakiagjS^'I'O 1 
Spaaklng 13 J 

?aading f 



*ricir.g 

Liscariing 
jpa^K.irr j 
V aiding 



35 
25 

:o 
:o 

3C-Encrv 
60-Sxlc' 1 
iO-Sncr" 
30-Sxtc 



• labia lookup of raw scora 
Tabla lookup of raw scora 
Tabla lookup of raw scora 

Tabla lookup of raw scora 
Racing scala of six lesaa, 
racad 1-3, than avsragsd. 
Scora convarcad by cab La lookup. 

Saaa as K-6 * 
Saaa as $-6 
Saaa as :<-o 
Saaa as K-6 
Saaa as K-o 

Lis caning x 2,5 « Liaxaaing 
Uiacaning/spaaking ♦ Spaakiag) 
x 2.3 • Spaaklng 

Ilona 

Lla caning/ ipaaking * Spaaklng 

■ S packing > 
Nona 

Nona.- 

Liscanlng * .3 ■ Liacanina; 

Nona. 

Raadtng x .6 . 
?.aadir.g r. .5 " ? *" di *« 

- : ,0 ? f t - Vrlelot 

-rlcing x *5 • 



:£ s:c:«sj v.cha'. 
£5 ?*, 



.125 
.123 

.123, 

.123 
.3 

.125 
.123 
.125 
.125 
.3 

37.5* 
62.5 



15 
25 

35 
25 

10 
20 
30 

40 





*-i2- wlscaniag , 


:o 


Lis caning < .3 


■ Llscaning 


10 




s^aa/lr.* 


20 


Nona 




10 




?«Hiag , 




Raiding x 


5 - 


Raiding 


30 




-ritlr.g 


*P 


Anting x . 


3 ■ 


Pricing 


i0 


Tea;* s ; 


• - • - ?rwar.:iacto?. 




Nona 






6 




"rarrar 




Nona 










V:ca?**ary 


*) 


Nona 






6 






6 


Nona 






6 






6 


Nona 




r 


6 




'I-.- ?.a,:a ?f l«ar\.l,ng. 


« 

J • 


Avaraga across 


ail eaacaars 
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Avariga icr 


?ai 


all :aa :hars > 






'•>:' :v.;v -abt'.s 




Avaraga air 


319 


all taaefcan ' 


• 




C?s3l*:a*«si *ir.d 






4. 








:lst*i-aas *? v-r* 


3 


Avaraga acr 


319 


all Caachari 


a 




2arci;:ra:'.on 


3 


Avaraja icr 




all :aicr.«:i • 


3 




Ail :j : rraur. l- 








t 






, ca:a >*.:.-. :ia;r.ar 


5 


Nac :oopuca 










- 3 . 1 1 : • : ; :;<35»jr,i- 








» 


» 




* virh paari 




Not coap'icad 
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EXHIBIT A- J 
Computation Algorithms for Total Score 



i 



tirades 



K-12 

K-2 

3-12 

K-12 

K-2 
i-12 

K-!2 



Assessment 
Time * 



Score Type 



Entry/Exit 
Kntry 
Entry - 

Kntry 

Exit 
Exit 

Kxit 



K-12 Entry/Exit 



LAS (a) 

MKC 

MKt; 

Teacher 
Evaluation 

MKC 

MKC 



Teacher 
Evai uation 



TOTAL 



Algorithm 



Maximum 
Score 



(.125+.125+.125+.125+.5) x 10 

(Listening+Speaking) x y .6 

(Listening-tSpeaking+Reading 
♦Writing) x ,6 

[No conversion necessary] 

(Listenlng+Speaking) x .5 

(Listcning+Speaking+kcading 
♦"Writing) x .5 



10 

60 

60 
30 

50 

50 



0>> 



(b) 



(Kate learn. ♦ acad. perf. + 
work/study skills + completeness * . . 
+ class par tic in.) x .4 AO 



particip 
LAS + MEC + Teacher Evaluation 



100 



(a) .125 and .5 are maximum table lookup scores. 

(I>) Weighting tact or was changed to .6 tor 1981-82- year. 

(c) Weighting i at tor was changed to .3 for 1981-82 year. 



MC 
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EXHIBIT A-4 

# 

ESOL /Bilingual Placement Levels as Determined 
By Entry/Exit Test Scores 




K-2 



3-6 



7-12 



1 

.2- 
6 

1 
2 
3 

6, 

1 

2 
3 
4 
5 
6 



Beginning 
Intermediate 
No ESOL 

Beginning 
Intermediate 
Advanced 
No ESOL 

Low Beginning 
High Beginning 
Low Intermediate 
High Intermediate 
Advanced 
No ESOL 



0- 54 
55- 74 
75-100 

0- 54 

55- 74 
75- 84 
85-100 

0- 40 
41- 55 

56- 64 
65- 74 
75- 84 
85-100 



589b/75 
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TABLE B-l 

Pairwise Correlations Between LAS and MEC Entry Scores 



< 

Score Pairs Grades K-2 Grades # 3-6 Grades 7-8 Grades 9-12 



LAS Total* .with MEC Total. 


.74 


.77' 


.75 


.74 - 


LAS Total with MEC 'Listening 


.74 


.78 


.71 


.73 


and Speaking Combined 










LAS 'Total with MEC Reading 


/-* / 


.66 


.69 


.70 


LAS Total with MEC Writing 




.66 


• 71 


.61 


-Number of Students in Sample 


725 


635 


242- . 


587 



In Grades K-2 the MEC contains only Listening and Speaking. 



TABLE B-2 

Pairwise Correlations Between LAS and MEC Exit Scores 



Score Pairs Grades K-2 Grades 3-6 Grades 7-8 Grades 9-12 



LAS Total with MEC Total 


,73 


.56 


.64 


.77 


LAS Total with MEC Listening 


.73 


.59 


. .70 


. 72 


and Speaking Combined 










LAS Total with MEC Reading 


_* 


..47 


.56 


.70 


LAS Total with MEC Writing 




.47 


.50 


.65 


Number of Students in Sample 


828 


716 


262 ' 


593 



In Grades K-2 the MEC contains only Listening and Speaking. 



589b 
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TABLE B-3 

T-Tests of Significance of Entry to Exit Gains for Students with Nonzero 

Entry and Exit Total Scores 1 





* 


Mean 


Mean 














Entry 


Exit 


Mean 








Grades 


Measure 


Score 


Score 


Gain 


T 


DF 


Probability 


K-2 


MEC Listening and 
















Speaking and MEC 
















Total 


19.1 


31.4 


12.3 


30.6 


476 


.000 




LAS 


- 4.6 


7.1 


2.5 


23.3 


476 


.000 




• Total Battery 


45.9 


70.7 


24.8 


26.2 


476 


.000 


3-b 


e 

MEC Listening and 
















Speakfng 


24.2 


35. 1 


10.9 


22.7 


423 


.000 


* 


MEC Reading 


12.0 . 


25. 1 


13. 1 


29.5 


423 


.ooa ■< 




MEC writing 


8.5 . 


15.6 


7.1 


21.5 


423 


.000 




MEC Total 


44.6 


75.7 


31.1 


30.1 


423- 


.000 




LAS 


5.5 


7.8 


2.3 


20.8 


423 


vOOO 




Total Battery 


45. 9 

* 


70.3 


* 24.4 


23. 1 


423 


♦ 000 


7-0 


MEC Listening and 


• 








- 






Speaking 


L4.4 


20. 1 


5.7 


12.7 


138 


.000 




MEC Reading 


13.2 


19.7 


6.5 


12.4 


138 


.000 




MEC Writing 


15.7 


23.9 


8.2 


10.0 


138 


.000 




MEC Total ' 


43.4 


63.9 


20.5 


13.7 


138 


.-000 




LAS 


5 0 


7 1 


2 1 


12 1 


1 ifl 

1 JO 






Total Battery 


44.2 


64.2 


20.0 


13.2 


138 


. .000 


9-12 


MEC Listening and 
















Speaking 


14.0 


19. 1 


5.1 


18.4 


353 


.000 , 




MEC Reading 


15.7 


2P.7 


5.0 


17.2 


353 


.000 




MEC Writing 


19.4 


25.8 


6.4 


1,5.9 


353 


.000 




MEC Total 


49.2 


65.7 


16.5 


22.7 


353 


.000 




LAS 


5.Q 


7.1 




18.0 


353 


v.000 




Total Battery 


47.2 


66.2 


19. a 


•23.3 


353 


.000 

1 
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TABLE B-4 

Results of MEC Testing of Native Englishrspeaking Students 



•r 



Grades Grades , Grades Grades 



Total | K-2 3-6 7t8 9-12 ' 

Score Range N % N Z ^.N X N Z N Z 



3 

Less Chan 80 


. / 






<> 














0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


80-84 b 


7 , 


4 


2 


f 

5 


1 


1 


4 


10 ' 


0 


0 


85-89 C 


15 


8 


3 v 


7 


3 


3 


6 


15 


3 


12 


90-94 c 


52 


27 


8 


18 


25 


29 


9 


23 


10 * 


38 


95-l00 C 


122 


62 


31 


70 


57 


66 


21 


53 


13 


50 



(a) Failing scote at all grade levels. 

(b) Passing score for Grades K-2; Jailing at all other grade levels, 

(c) Passing score at all grade levels. 

1 / 
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Mean Standardized Entry Test Scores by ESOL Proficiency Level* 



Grades Instrument 


Level 1 Level 2 Level 3 Level 4 Level 5 


level* 6 

(No ESOL) 


K-2- LAS . ' 


.38 i 


A 5 


1.00 




1.11- 


MEC ' 


- -.43 / 


.65 


1.13 




1.25 ; 


Teacher Evaluation 


- .43/ 


.56 


1.20 . 




,1.50 


Total 


- .44/ 


• .66 


1.20 




1.38 


Number of Students 


565 


205 


75 '. 




59' 



3-6 


LAS 


/ .54 


.55 


* .85 


4 




1.06 




MEC 


r - 74 


.,61 


1.09 






1.4! 




Teacher Evaluation^ 


1 - ' . 64 


.54 


"Irfl 






1.46 




Total / 


- .72 


.61 


1 .09 






1.46 




Number of Students 


"289 


215 


95 






34 
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7-8 


LAS <i / 


- .62 


.15 


.53 


.86 


.99 


1.14 




MEC / 


- .81 


.24 


.68 


.95 


1.23 


1.51 




Teacher Evaluation 


- .80 


.06 


.57 


.82 


1.07 


1.77 


• 


Total ■ 


- .79 


.22 


.70 


.98 


.1.25 


l.6"7 




. • i 

Number of Students 

< 


87-. 


54 


44 


- 35 


14 


8 






\ - 
















V 










9-2 


LAS 


-' .82 - 


.31 


.47 


.77 


.91 


1.18 




MEC 


- 1.05 - 


.17 


.43 ■ 


.83 


1.12 " 


1.19 




Teacher Evaluation 


- 1. 00 - 


.27 


.40 


\76 


1.16 


.1.72 




Total 


- 1.07 


.23 


.45 


.86 


1.19 * 


1.44, 




Number of Students 


149 

t 


140 


124 

lin 


75 


84 . 


15 . 


*Proficiency Levels: 




■\ 






1 





K-2: 1-Begirining; 2«Intertaediate; 3«Unknown (thera should be no* iev.al 3, 
students in grades K-2, yet 75 were included in the ejitry test 
population). 

3-6: l«3eginning; 2-Intern.ediate; 3«Advanced. 

7-12: l«Low beginning; 2*High beginning; 3-Low intermediate; 4«High 
intermediate; 5*Advaticed. / ^ 
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\ TABLE B-6 * ' • ' ' ' ' . 

Standard. Deviations and Standard Errors of Measurement 
^-p . of Entry and Exit Total. Scores by 'Grade Levels 







1 ■ 


> 


















"Entry. 




Exit-. 




Standard 


Standard Error* 


Standard 


.Standard Error* 




Deviation 


of Measurement ' 


Deviation 


of Measurement 


R-2 


". 28.8 


13.0' 


. *16.0 


: •* 7.2 • 


3V6 . ' 


-'28.2 -* 


12.7 


15.5 


7.0 


7-6 


J 25.6 . 


11.5 


15.0 - 


6.8 


9-12 


-4 22V4 " 


- ,. 10.1 


-14.4 

* 


■': 6.5 



^ ^The standard errors of measurement: are computed On the basis of * an assumed 
, -' total- Score reliability of .8. If the true total £coVa reliability is lower, 
the -V st andard ^errors/ of measurement would be, JargeF. The ■ formula ''id'. 
SEM*SD'J l~r if here SEM=»Standard Error of Measurement, SD*Standard Deviation; 
and ^Reliability Coefficient. , . / • \ § 
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TABLE B-7 ' - .' 

'ESOL /Bilingual Flacemenfc'Levels as Determined 
"By Entry/Exit Test Scores 



Grades 


Level 


'" Description 


Entry/Exit Test^\ 
* .Score Range 




K-2 


1 


Beginning . 


0-54 






2 


1 Intermediate 


" 55- 74 






6 


No ESOL . • 


75-100 


* 


3-6 
- m 


. 1 


Beginning* 


' .. 0- 54 






2 


Intermediate 


55- 74 . 






3 


Advanced 


75- 84 . . 






6 


Ho ESOL 


85-100 


> 


v 7-12 


1 


Low Beginning 


0- 40 






.2 


High Beginning 


41- 55. 


• \ . 




3 


Low Intermediate . 


'. , 56- 64. 






L 4 


High Intermediate 


" 65- 74 






* 5 


Advanced , 


75- 84 






6 


No ESOL 


85-100 
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B-6. 



Sta&dtrjliztd 
Scdra 
Means 



,1.6 



1.2 



.8 



.4 



-.4 



-,8 



•1.2 



-1.6 



ENTRY 



EXIT 



' FIGURE &-1 . 

Entry and Eric Maan LAS and MEC cor as by Laval - Gradas K-2 




MEC Tot./ 

^ LAS 

* • MEC Tot./ v 
Llsc.+Spkg. 

LAS 



Lagand 



LAS 



>. v 



— • 4 — MEC TocAl 

List.+Spkg. 
....... Raadidg 

MEC Writing 



Beginning 



Incermediaca 
8*7 



ERjC y 



Unknown No ESOL lava! ig/ ' 

Program 



43 mm mum 



Scandarfrizad 
Scora 
Huns 



.1-4 



1.0 



.6 



-.2 



-.6 



-1.0 



-1.4 



-1.3 



ENTRY 



EXIT 



PJGURB E-2 

t ... • • . 

Entry and Exit: Mean LAS and MEC Scores by Laval - Gcadss 3-6 



MKC Toesl 



,MIC Wciclnt 




^ NEC Use.+Spkg. 
US 



. NEC ToGftl . 
KEC Writing 
M£C X*ft41ftf 
KEC LUC.^kf. 



us 

MEC Total 



MEC RMdiQf 
HEC Writing 



Beginning 



9 

ERJC 



[inttnMdiAtt 



Advanced 



6 

No ESOL 



Uvti in 
£rogr«a 



cist Ciia AvuiiiaE 



Standardized 
Score 
Means 



1.4 



1.0 



.6 



-.2 



-.6 



-1.0 



-1.4 



-1 ft 



FIGURE B-3 

Entry and Exit Mean LAS and MEC Scores by Level - Grades 7-8 

• . •- ' ■ •• i . ' . - 

HSC Tac«J 
V*y HEC Wciclns 

• • 

' V" V HEC Total • 



ENTRY 



EXIT 




HEC Toc*i 
WCC U*4ia« 

LAS 



Legend 



LAS 



— MEC Total . 
• • - • List.+Spkg. 
••••••• MEC Reading 

a^-v-^v MEC Writing 



-/- 



B.gloning 



3 4 
Zoc.rm.diac. 



5 6 L.v.l la 

Adv»nc«d Ho ESOL Program 



9 



1-9 



45 



BESI Bid lutiiuiBU 



Standarfrlzad 
Scort 
Mtan* 



FIGURE B-4 

■ •>, 

Entry and Exit Hus LAS and MEC Scort* by £«v*l - GradM 9-12 



1.4 



1.0 



.2 



-.2 



-.6 



-1.0 



-1.4 
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ENTRY 



EXIT ' 




Llsc.+Spfcg. 



NIC local 
US 



NIC Tactl 
HCC utlelM 
Utc.Hpkg. 
HKC UUUt 

mc uncus 

LAS/ „ 



Total V* 
Liic.+Spkg. 

tfriciag 



Beginning < 
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